Detection and Summarization of Novel Network Attacks Using Data Mining

This paper introduces the Minnesota Intrusion Detection System (MINDS), which uses a suite of data mining techniques to automatically detect attacks against computer networks and systems. While the long-term objective of MINDS is to address all aspects of intrusion detection, in this paper we present two specific contributions. First, we present MINDS anomaly detection module that assigns a score to each connection that reflects how anomalous the connection is compared to the normal network traffic. Experimental results on live network traffic at the University of Minnesota show that our anomaly detection techniques have been successful in automatically detecting several novel intrusions that could not be identified using state-of-the-art signature-based tools such as SNORT. Many of these have been reported on the CERT/CC list of recent advisories and incident notes. We also present the results of comparing the MINDS anomaly detection module to SPADE (Statistical Packet Anomaly Detection Engine), which is designed to detect stealthy scans. Second, we present MINDS association pattern analysis module that summarizes those network connections that are ranked highly anomalous by the anomaly detection module. Given the very high volume of connections observed per unit time, such characterization of novel attacks is essential in enabling a security analyst to understand emerging threats. Experimental evaluation shows that the MINDS approach is very useful in creating accurate summaries of novel attacks.